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A  DUAL  MAXIMUM  PRINCIPLE  FOR  DISCRETE-TIME  LINEAR  SYSTEMS 
WITH  ECONOMIC  APPLICATIONS 


C.  Duncan  MacRae 

Abstract  -  A  discrete-time  linear  optimal  control  problem  with  given  initial 
and  terminal  times,  state-control  constraints,  and  variable  end  points  is  set 
forth.   Corresponding  to  this  primal  control  problem,  or  maximization  problem, 
is  a  dual  linear  control  problem,  or  minimization  problem.   The  fundamental 
properties  of  duality  are  stated.  A  dual  maximum  principle  is  proved  with 
the  aid  of  the  duality  theory  of  linear  programming,  where  the  dual  of  the 
Hamlltonian  of  the  primal  control  problem  is  the  Hamiltonian  of  the  dual  control 
problem.   A  discrete-time  analogue  of  the  Kami 1 ton- Jacob 1  equation  is  derived; 
and  economic  applications  are  discussed. 


INTRODUCTION 

One  of  the  major  developments  in  modern  control  theory  Is  the  maximum 
principle.   Although  it  was  developed  by  Pontryagin  [1]  for  continuous-time 
problems,  it  has  been  extended  to  discrete-time  problems  by  Halkin  [2,3], 
Holtzman  [4,5]  Jordan  and  Polak  [6]  through  a  geometric  approach,  and  by 
Pearson  and  Sridhar  [7],  Mangasarian  and  Fromovitz  [8],  and  Arimoto  [9]  with 
mathematical  programming  theory. 

Another  development  of  importance  is  the  concept  of  duality  in  optimal 
control  problems.  The  notion  of  duality  was  first  discussed  in  modern  control 
theory  by  Bellman  [10]  in  the  analysis  of  a  continuous-time  bottleneck  process. 
He  formulated  a  dual  process  both  to  verify  a  proposed  solution  to  the  al- 
location problem  and  to  analyze  the  nature  of  the  optimal  process.  His  re- 
sults have  been  extended  by  Tyndall  [11]  and  Levinson  [12]  for  a  class  of 
continuous  linear  programming  problems.   Kalman  [13,14]  observed  duality  in  a 
different  context.   He  recognized  that  the  linear  filtering  problem  is  the 
dual  of  the  linear  regulator  problem.   Observability  and  controllability  are 
dual  concepts.   His  analysis  has  been  generalized  by  Pearson  [15]  for  con- 
tinuous-time systems.   Pearson  [16],  Hanson  [17],  and  Ringlee  [18]  have 
exploited  the  idea  of  duality  to  derive  a  lower  bound  for  a  convex  variational 
programming  problem.   Following  the  formulation  of  necessary  conditions  for  the 
solution  of  the  problem  by  Berkovitz  [19],  they  derive  a  dual  problem  which 
yields  the  lower  bound  given  a  feasible  solution  to  the  primal  problem.   The 
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knowledge  of  the  lower  bound  Is  helpful  In  the  computation  of  the  solution. 
Pearson  [20]  and  Kreindler  [21]  have  analyzed  reciprocity  and  duality  in  control 
programming  problems.   The  reciprocal  or  dual  of  a  primal  control  problem  is 
obtained  by  interchanging  the  role  of  the  state  and  co-state  variables  along  the 
extremal  of  the  primal.   The  statement  of  the  reciprocal  problem  contains  both 
state  and  co-state  variables.   Pearson  distinguishes  the  dual  problem  from  the 
reciprocal  problem  by  the  condition  that  the  extremal  of  the  primal  problem  is 
an  extremal  of  the  dual  problem.  Kreindler  refers  to  the  dual  problem  only  as 
a  reciprocal  problem,  since  the  reciprocal  of  the  reciprocal  Is  not  the  primal. 
He  shows  that  along  an  extremal  the  Hamiltonian  of  the  primal  problem  with 
inequality  constraints  is  equal  to  the  Hamiltonian  of  a  reciprocal  problem 
without  inequality  constraints.   Pearson  [22],  Pallu  de  la  Barriere  [23],  Mond 
and  Hanson  [24]  have  further  discussed  duality  in  continuous- time  systems. 
With  the  exception  of  Kalman's  work  the  discussion  of  duality  in  modern  control 
theory  has  been  limited  to  continuous-time  systems.  Moreover,  with  the  ex- 
ception of  Kreindler's  result  there  has  been  no  examination  of  the  relationship 
between  the  Hamiltonian  of  the  primal  control  problem  and  the  Hamiltonian  of 
the  reciprocal  or  dual  control  problem. 

The  primary  contribution  of  this  paper  is  a  dual  maximum  principle  for 
discrete-time  linear  systems.   This  theorem  expresses  the  relationship  between 
duality  and  the  maximum  principle  in  linear  control  problems  and  provides 
necessary  and  sufficient  conditions  for  an  optimal  solution.   Moreover,  it  has 
an  immediate  economic  Interpretation  in  terms  of  allocation  and  valuation  of 
resources  over  time.   The  proof  of  the  theorem  follows  from  the  complementary 
slackness  theorem  and  the  duality  theory  of  linear  programming. 


The  application  of  linear  programming  to  optimal  control  problems  is  not  new. 
See  Zadeh  [25],  Whalen  [26],  Dantzig  [27],  and  Rosen  [28]. 


DUALITY  THEORY 

In  this  section  we  examine  duality  in  linear  control  problems.  Consider 
a  discrete- time  linear  control  problem  with  given  initial  and  terminal  times, 
state-control  constraints,  and  variable  end  points. 

Primal  Linear  Control  Problem 

Choose  s-  -  0  and  u(k)  ^  0  for  k  =  0,  1,...,N  -  1,  so  as  to  maximize 

t     N-1  t 

J  -  if.3^   +  E  [a'(k)x(k)  +  b'(k)u(k)]  +  f^x(N)  (1) 

"  "   k-0  " 

subject  to                  GqSq  +  x(0)  =  hQ,  (2) 

D(k)u(k)  -  F(k)x(k)  <  d(k),   k  =  0,  1, . . . ,N  -  1          (3) 

Ax(k)  -  A(k)x(k)  +  B(k)u(k)  +  c(k+l),   k  =  0,  1, . . . ,N  -  1          (A) 

and                          G^x(N)  <  hj^,  (5) 

where  s^  is  a  vector  of  primal  pre-control  variables,  x(k)  is  a  vector  of  primal 
state  variables,  u(k)  is  a  vector  of  primal  control  variables,  f^.,  f„,  a(k), 
b(k),  c(k+l),  d(k),  hp,  and  h   are  given  vectors,  A(k) ,  B(k) ,  D(k),  F(k) ,  Gq, 
and  G  are  given  matrices,  Ax(k)  =  x(k+l)  -  x(k),  0  is  the  given  primal  initial 
time,  N  is  the  given  primal  terminal  time,  and  x(k)  is  taken  as  given  in  (3). 

The  general  formulation  of  the  discrete-time  linear  control  problem  given 
in  (1)  -  (5)  allows  for  fixed  or  free  end  points,  equalities  in  the  constraints, 
and  pure  control  or  state  constraints.   Although  x(k)  is  taken  as  given  in  (3), 
we  can  allow  for  primal  state  constraints  of  the  form 

M(k)x(k)  S  n(k)      k  -  1,  2,...,N  -  1  (6) 


by  substituting  (4)  into  (6)  to  obtain 

{M(k+l)B(k)}u(k)  -  {-M(lcfl)[I  +  A(k)]}x(k)  <   {n(k+l)  -  M(k+l)c(k+l)} 

k  =  0,  1,...,  N  -  2  (7) 

which  is  of  the  form  given  by  (3) .  Note  that  state  constraints  from  period  1 

to  period  N  -  1  imply  constraints  on  the  control  variables  only  up  through 
period  N  -  2.   If  the  constraints  on  the  initial  primal  state  are  of  the  form 
(6)  rather  than  (2) ,  they  can  be  represented  by  beginning  the  primal  problem  at 
k  -  -1  with  a  free  state. 

A  feasible  solution  to  the  primal  linear  control  problem  is  a  sequence 
{x(k),  u(k);  Sq}  satisfying  (2)  -  (5)  with  u(k)  and  s^  I   0.  Let  J{x(k) ,  u(k); 
Sq}  be  the  value  of  J  for  the  feasible  solution  {x(k),  u(k);  s^}.   An  optimal 
solution  to  the  primal  control  problem  is  a  feasible  solution  {x(k),  u(k); 
s-}  such  that  _   - 

J{x(k),  u(k);   Sq}  >  J{x(k),  u(k);  s^}  (8) 

for  all  feasible  solutions  {x(k),  u(k);  s-} . 

The  primal  linear  control  problem  can  be  reformulated  as  a  linear  pro- 
gramming problem. 

Primal  Linear  Programming  Problem 

Find  8q  >  0,  x(k)  unrestricted  for  k  -  0,  1,...,N,  and  u(k)  ^0  for 

k  -  0,  1,,..,N  -  1  which  maximize 

N-1 
J  -  f_8.  +  Z   [a'(k)x(k)  +  b'Ck)u(k)]  +  fAxCN)  (9) 

k-0  " 

subject  to  GqSq  +  x(0)  -  h^,  (10) 


-F(k)x(k)  +  D(k)u(k)  <  d(k)    k  =  0,  1,. . . ,N  -  1,  (11) 

-[I  +  A(k)]x(k)  -  B(k)u(k)  +  x(k+l)  =  c(k+l)    k  -  0,  1,...,N  -  1,   (12) 
and  Gj^x(N)  -  l^-  ^^^^ 

Tucker  Diagram 

The  structure  of  the  primal  linear  programming  problem  can  be  seen  more 
clearly  with  the  aid  of  the  detached  coefficient  array  developed  by  A.  W.  Tucker  ' 
[29],  The  primal  variables  are  listed  across  the  top  of  the  diagram  along 
with  their  sign  restrictions.  Each  row  of  the  matrix  gives  the  coefficients  of 
one  of  the  constraints,  while  the  rightmost  column  gives  the  constants  of  the 
right-hand  side  of  each  constraint.  The  bottom  row  gives  the  coefficients  of  the 
primal  variables  in  the  objective  function. 

Corresponding  to  the  primal  linear  programming  problem  is  a  dual  linear 
programming  problem.  The  dual  variables  are  listed  in  the  left-most  column 
and  the  dual  constraints  are  read  down  the  columns  of  the  matrix.  The  right- 
most column  gives  the  coefficients  of  the  dual  objective  function. 

Dual  Linear  Programming  Problem 

Find  Yjj  -  0,  H'(k)  unrestricted  for  k  «»  N,  N-1, . . .  ,0,and  X(k)  ^  0  for 
k  «»  N-1,  N-2,...,0,  which  minimize  ' 

0 


f 


$  -  hjjYjj  +  I     [d'(k)X(k)  +  c'(k+l)4'(k+l)]  +  hQV(O)  (14) 
k"N  1 

subject  to  Gj^Yf,  +  'i'(N)  =  f^,  (15) 

D'(k)X(k)  -  B'(k)Y(kfl)  >   b(k)   k  =  N-1,  N-2,...,0,  (16) 
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VCk)  -  F'(k)X(k)  -  [I  +  A'(k)]S'(k+l)  =  a(k)   k  -  N-1,  N-2 0,    (17) 


and  Gq^CO)  >  f^.  (18) 


The  dual  linear  programming  problem  can  be  reformulated  as  a  dual  linear 
control  problem  with  given  initial  and  terminal  times,  state-control  constraints, 
and  variable  end  points. 

Dual  Linear  Control  Problem 


Choose  YjT  -  0  and  X(k)  -  0  for  k  =  N-1,  N-2,...,0  so  as  to  minimize 

*  "  Vn  +   ^  [c'(k+l)V(k+l)  +  d'(k)A(k)]  +  h.H'(O)       (19) 
k-N-1  ' 


^'"'^'""        .       G;y^  +  Wf,  (20) 


D'(k)X(k)  -  B'(k)H'(k+l)  ^  b(k)   k  =  N-1,  N-2,...,0,       (21) 
A4'(k+1)  -  A'(k)4'(k+1)  +  F'(k)A(k)  +  a(k)   k  -  N-1,  N-2,..., 0,      (22) 


and  G^H'(O)  >  f^,  (23) 


where  H'(k+1)  is  a  vector  of  dual  state  variables,  X(k)  is  a  vector  of  dual 
control  variables,  y     is  the  vector,  of  dual  pre-control  variables,  A4'(k+1)  = 
4'(k)  -  ^(k+l),  N  is  the  dual  initial  time,  0  is  the  dual  terminal  time,  and 
4'(k+l)  is  taken  as  given  in  (22).^ 


2 
Note  that  the  statement  of  the  dual  linear  control  problem  contains  only  dual 

variables.   In  general,  this  will  not  be  true  for  nonlinear  control  problems. 

See  Pearson  [20]  and  Krelndler  [21]. 


A  feasible  solution  to  the  dual  linear  control  problem  is  a  sequence 
{Ydc+D,  X(k);  Yjj}  satisfying  (20)  -  (23).   Let  ^{^(k+l),  X(k);  Yj^}  be  the 
value  of  <J>  for  the  feasible  solution  {4'(k+l) ,  X(k);  Yjj}«   An  optimal  solution 
to  the  dual  linear  control  problem  is  a  feasible  solution  {^(k+l),  X(k);  y^i 
such  that 

*{?(k+l),  X(k);  Yjj}  -  ^{^(k+l),  X(k);  y^}  (24) 


for  all  feasible  solutions  {4'(k+l),  X(k);  Yj,}. 


Fundamental  Properties  of  Duality 

The  primal  (dual)  problem  is  a  maximization  (minimization)  problem.   The 

primal  (dual)  system  moves  forward  (backward)  in  time;  and  the  primal  initial 

3 

(terminal)  time  is  the  dual  terminal  (initial)  time.   Moreover,  the  dual  of  the 

dual  control  problem  is  the  primal  control  problem. 

Theorem  1  (Boundedness  Theorem).   If  {x(k),  u(k);  s_}  and  {4'(k+l) ,  X(k); 

Y„}  are  feasible,  then 

N 

J{x(k),  u(k);  Sq}  <  (D{H'(k+l),  X(k)-,  Yjj>  •  (25) 


Theorem  2  (Unboundedness  Theorem) .   If  the  primal  (dual)  linear  control 
problem  is  feasible,  but  the  dual  (primal)  linear  control  problem  is  not 
feasible,  J  (<I))  is  unbounded  from  above  (below). 

Theorem  3  (Existence  Theorem) .   The  primal  (dual)  linear  control  problem  has 
an  optimal  solution  if  and  only  if  both  control  problems  have  feasible  solutions. 


3 
This  property  of  duality  was  first  observed  by  R.  Bellman  [10]  in  the  analysis 

of  a  continuous- time  bottleneck  process.   R.  E.  Kalman  [13,14]  also  perceived 

this  property  in  his  study  of  the  linear  filtering  problem. 
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Theorem  A  (Duality  Theorem).  A  primal  (dual)  feasible  solution  {x(k),  u(k) ; 
8-}  ({y(k+l),  X(k);  Yjj))  is  an  optimal  solution  if  and  only  if  there  exists  a 
dual  (primal)  feasible  solution  {?(k+l) ,  X(k) ;  y^}    ({x(k),  u(k);  Sq})  with 

J{x(k),  G(k);  8q}  =  <I>{nk+l),  X(k);  Yn>-  (26) 


Theorem  5  (Complementary  Slackness  Theorem) .  A  primal  (dual)  feasible 
solution  {x(k)  ,  u(k);  s.}  ({^(k+l) ,  X(k);  Yv,})  is  an  optimal  solution  if  and 
only  if  there  exists  a  dual  (primal)  feasible  solution  {'i'(k+l),  X(k);  Y«J 
({x(k),  X(k);  Sq})  with 

X'(k)[D(k)G(k)  -  F(k)x(k)  -  d(k)]  =0   k  =  0,  1,...,N-1,       (27) 

G'(k)[D'(k)X(k)  -  B'(k)$(k+1)  -  b(k)]  -  0   k  -  N-1,  N-2,...,0,   (28) 

and  the  transversality  conditions 

VO  "   °'  <29) 

and  9jjSjj  =  0,  (30) 


where 


Yq  "  V^°^  -  ^0  ^"^  ^N  *=  \  -  S'^^^^-  ^^^^ 


Theorems  1-5  are  derived  from  the  corresponding  theorems  of  linear  pro- 
gramming [29] . 

Theorem  5  is  a  statement  of  thir*  Kuhn-Tucker  [30]  necessary  and  sufficient 
conditions  for  an  optimum  solution.  Note  that  the  initial  (terminal)  trans- 
•ersality  condition  for  the  primal  control  problem,  (29) ((30)),  is  the  terminal 
(initial)  transversality  condition  for  the  dual  control  problem. 

Theorem  6  (Sensitivity  Theorem).   If  {x(k) ,  u(k);  Sq}  and  {^(k+l) ,  X(k); 
Y„}  are  optimal. 
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V^^j^jj{x(k)}  -  4'(k)   k  »  0,  1,...,N,  (32) 

and  V^.j^.<I>{¥(k)}  -  x(k)   k  =  N,  N-1,...,0  (33) 

where  V  ..  .j{x(k)}  Is  the  gradient  of  J  with  respect  to  x(k)  evaluated  on 
{x(k),  u(k);  8^}  if  the  gradient  exists,  etc. 

The  proof  of  (32)  and  (33)  follows  immediately  from  the  sensitivity  theory 
of  linear  programming  [31],   In  (32)  and  (33)  x(k)  and  ^(k)  are  treated  as 
parameters.  Note  that  the  derivatives  in  (32)  and  (33)  may  be  one-sided. 

Although  the  sensitivity  analysis  is  in  terms  of  derivatives,  we  can  be 
sure  for  finite  changes  in  the  parameters  that  (32)  and  (33)  describe  upper 
bounds  on  the  change  in  J  and  lower  bounds  on  the  change  In  $  since  J  is  concave 
in  the  parameters  and  $  is  convex. 


HAMILTONIAN  THEORY 

In  this  section  we  investigate  the  relationship  between  duality  in  linear 
control  problems  and  the  corresponding  Hamiltonian  programming  problems. 
Consider  a  sequence  of  primal  linear  programming  problems  for  k  -  0,  1 N-1, 

Primal  Hamiltonian  Programming  Problem 
Choose  u(k)  -  0  so  as  to  maximize 

7/(k)  -  a'(k)x(k)  +  b'(k)u(k) 
+  'i"(k+l)[A(k)x(k)  +  B(k)u(k)  +  c(k+l)]  (3A) 

I 

subject  to  D(k)u(k)  -  F(k)x(k)  ^  d(k),  (35) 
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where  x(k)  and  ^(k+l)  are  given,  ^/Ck)  is  the  Hamiltonian  of  the  primal  linear 
control  problem,  and  (35)  describes  the  state-control  constraints  of  the  primal 
control  problem. 

A  feasible  solution  to  the  primal  Hamiltonian  programming  problem  is  a 
vector  u(k)  >  0  satisfying  (35).  Let^[u(k);  x(k)  ,  ^(k+l)]  be  the  value  of  ^ 
for  the  feasible  solution  u(k),  given  x(k)  and  V(k+1).  An  optimal  solution 
to  the  primal  Hamiltonian  program  is  a  feasible  solution  u(k)  such  that 

[G(k);  x(k),  4'(k+l)]  >  'Mu(k);  x(k),  ^(k+l)  ]  (36) 


for  all  feasible  solutions  u(k),  given  x(k)  and  ^(k+l). 

Corresponding  to  this  sequence  of  primal  Hamiltonian  programming  problems 
is  a  sequence  of  dual  linear  programming  problems  for  k  *•   N-1,  N-2,...,0. 

Dual  Hamiltonian  Programming  Problem        ;. 

Choose  X(k)  -  0  so  as  to  minimize 

7({k)   «  c'(k+l)4'(lcfl)  +d'(k)X(k) 
+  x'(k)[A'(k)4'(k+l)  +  F'(k)X(k)  +  a(k)]  (37) 

subject  to  D'(k)X(k)  -  B'(k)f(k+1)  ^  b(k),  (38) 

where  ^(k+l)  and  x(k)  are  given,  Tftk)  is  the  Hamiltonian  of  the  dual  linear 
control  problem,  and  (38)  describes  the  state-control  constraints  of  the  dual 
control  problem. 

A  feasible  solution  to  the  dual  Hamiltonian  programming  problem  is  a 
vector  X(k)  >  0  satisfying  (38).  Let}^[X(k);  ^(k+l),  x(k)]  be  the  value  of 
/^(k)  for  the  feasible  solution  X(k),  given  ^(k+l)  and  x(k).  An  optimal  solution 
to  the  dual  Hamiltonian  programming  problem  is  a  feasible  solution  X(k)  such 
that 
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]^[\(k)-J(k+l),   x(k)]   <  /('[X(k);Y(k+l),  x(k)]  (39) 

for  all  feasible  solutions  X(k),  given  TCk+l)  and  x(k). 

Dual  Maximum  Principle 

Lemma  1.  A  feasible  solution  u(k)  (A(k))  to  the  primal  (dual)  Hamiltonian 
programming  problem  is  an  optimal  solution  if  and  only  if  there  exists  a  feasible 
solution  A(k)  (u(k))  to  the  dual  (primal)  Hamiltonian  problem  with 

X'(k)[D(k)G(k)  -  F(k)x(k)  -  d(k)]  "  0  (40) 

and  G'(k)[D'(k)X(k)  -  B'(k)H'(k+l)  -  b(k)]  -  0.  (41) 

Corollary  i. 

'y[G(k);x(k),  4'(k+l)]  =  ;^[A(k);Y(k+l),  x(k)].  (42) 

Lemma  1  and  Corollary  1  are  direct  applications  of  the  complementary 
slackness  and  duality  theorems  of  linear  programming  [29]. 
The  main  result  of  this  paper  can  now  be  stated. 

Theorem  7  (Dual  Maximum  Principle).  A  feasible  solution  {x(k),  u(k);  s_} 
({Y(k+1),  X(k);  Y  })  to  the  primal  (dual)  linear  control  problem  is  an 

A 

optimal  solution  if  and  only  if  there  exists  a  feasible  solution  {Y(k+1), 

X(k);  y  }    ({x(k),  u(k) ;  s_})  to  the  dual  (primal)  control  problem  with 

'\ 

^lu(k);   x(k),  <?(k+l)]  =;J^[X(k);  ?(k+l),  x(k)]        (43) 
and  satisfying  the  primal  and  dual  transversality  conditions,  (29)  and  (30), 

where  Ax(k)  -  Vy^j^^^j3^a(k)  ;x(k)  ,  y(kfl)]  (44) 


lA 


and 


Ank+1)  -  7^^j^j^[X(k);nk+l),  x(k)].  (45) 


The  proof  follows  from  Theorem  5,  Lemma  1,  and  Corollary  1. 

Hamllton-Jacobi  Inequality- 

We  can  use  the  dual  maximum  principle  to  derive  a  discrete-time  analogue 
of  the  Hamllton-Jacobi  partial  differential  equation  for  continuous- time 
optimal  systems. 

Theorem  8  (Hamllton-Jacobi  Inequality).   If  {x(k),  u(k);  s^}  and 
{^(k+l),  X(k);  Y„>  are  optimal  solutions  to  the  primal  and  dual  linear  control 
problems, 

J^{x(k)}  -  Jj^^{x(k)}  >'5!/[G(k);  x(k),  Y(k+1)] 

-/^[X(k);  nk+1),  x(k)]  >  $j^^j^{?(k+l)}  -  $j^{nk+l)}        (A6) 

where  J,  {x(k)}  is  the  optimal  value  of  J  at  the  beginning  of  period  k  with 
initial  primal  state  x(k)s,  etc.,  and  $,  . {'?(k+l)}  is  the  optimal  value  of  $ 
at  the  end  of  period  k  with  an  Initial  dual  state  Y(k+1),  etc. 

The  proof  of  the  theorem  employs  the  principle  of  optimallty  [10],  the 
mean  value  theorem,  and  Theorem  7.  Applying  the  principle  of  optimallty  to  the 
primal  and  dual  linear  control  problems  we  obtain 

Jj^{x(k)}  -  a'(k)x(k)  +  b'(k)G(k)  +  Jj^j^{x(k+1)}  (A7) 

or  Jjx(k)}  -  Jj^^j^{x(k)} 

-  a'(k)x(k)  +  b'(k)G(k)  +  J^^j^{x(k+1)}  -  J^^^{x(k)},       (48) 

and  *^j^{$(kfl)}  -  c' (lH-l)Y(k+l)  +  d'(k)X(k)  +  ^^{nk)}        (49) 
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or  *j^^{Y(k+l)}  -  *j^{y(k+l)} 


c'(lrt-l)Y(k+l)  +  d'(k)X(k)  +  $j^{Y(k)}  -  «I>j^{4'(kfl)}.     (50) 


From  the  mean  value  theorem  we  know  there  exists  a  6(k)  and  a  a(k)  subject  to 
0  ^  e(k)  5  1  and  0  ^  a(k)  ^  1  such  that 


Vj^^^^jj{x(k+1)  -  [1  -  e(k)]Ax(k)}«Ax(k) 


Jj^^{x(k+1)}  -  Jj^^^{x(k)}  (51) 


and  Vy.j^.*{4'(k)  -  [1  -  o(k)  ]A»i'(k+l)}'AY(k+l) 

-  «I»j^{nk)}  -  $J$(lcfl)},  (52) 

*"  V^(j^^jJ{x(k)}'Ax(k)5  J^^j{x(k+1)}  -  J^^^{x(k)}  (53) 

and  Vy^j^j(D{$(kH-l)}»A?(k+l)^  \{nk)}  -  *j^{Y(k+l)}  (54) 

since  J  is  concave  in  x  and  *  is  convex  in  Y,  where  V  .,  . >j{x(k+l)  -  [l-e(k)] 
Ax(k)}  is  the  gradient  of  J  with  respect  to  x(k+l)  evaluated  at  x(k+l)  - 
[l-e(k)]Ax(k)  if  it  exists,  etc. 

Substituting  (32),  (3A),  and  (53)  into  (48),  and  (33),  (37),  and  (54)  into 
(50),  we  obtain  (46)  from  (42),  (48),  and  (50).   Since  time  is  a  discrete 
variable,  (46)  is  an  Inequality  rather  than  an  equality. 


16 


ECONOMIC  APPLICATIONS 

The  problem  of  allocating  goods  and  services  over  time  can  be  formulated  as 
a  primal  linear  control  problem  with  the  aid  of  linear  activity  analysis  [32,33]. 
The  flow  of  commodities,  y(k),  is  assumed  to  be  a  linear  function  of  the  activity 
levels,  u(k): 

y(k)  -  C(k)u(k),  (55) 

where  C(k)  is  a  given  matrix  describing  the  technology  of  the  system.  The 
primal  state  variables  are  the  stocks  of  commodities,  x(k);  and  the  primal 
control  variables  are  the  activity  levels,  u(k) .   The  primal  control  problem  is 
to  choose  the  activity  levels  over  time  so  as  to  maximize  net  present  value,  j 
(1) ,  subject  to  the  initial  and  terminal  conditions  on  the  stocks  of  com- 
modities, (2)  and  (5),  the  capacity  constraints,  (3),  and  the  transformation 
of  stocks,  (4). 

Corresponding  to  the  allocation  problem  is  a  dual  problem  or  valuation 
problem.   The  dual  state  variables  are  the  imputed  prices  of  the  stocks  of 
commodities,  H'(k) ;  and  the  dual  control  variables  are  the  imputed  prices  of 
capacity,  X(k).  The  dual  control  problem  is  to  choose  the  capacity  prices 
over  time  so  as  to  minimize  the  imputed  cost  of  commodities  and  capacities, 
(19),  subject  to  the  initial  and  terminal  constraints  on  commodity  prices,  (20) 
and  (23),  the  no  unimputed  revenue  constraints,  (21),  and  the  transformation 
of  prices,  (22). 

The  economic  interpretation  of  the  dual  maximum  principle,  therefore,  is 
that  the  long  run  allocation  and  valuation  problems  can  be  decomposed  into  a 
sequence  of  short  run  allocation  and  valuation  problems,  or  primal  and  dual 
Hamiltonian  programming  problems.   The  primal  programming  problem  is  to  choose 
activity  levels  so  as  to  maximize  net  income  in  terms  of  revenue,  (34),  subject 
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to  the  capacity  constraints,  (35),  given  commodity  stocks;  and  the  dual 
programming  problem  Is  to  choose  capacity  prices  so  as  to  minimize  net  Income 
In  terms  of  Imputed  cost,  (37),  subject  to  the  no  unlmouted  revenue  constraints, 
(38),  given  commodity  prices.   The  Hamilton-Jacobl  Inequality,  then,  shows 
the  way  the  allocation  sequence  moves  forward  In  time  describing  the  Impact  of 
pricing  decisions  tomorrow  on  Imputed  cost  today  and  the  valuation  sequence 
moves  backward  In  flme  describing  the  Impact  of  allocation  decisions  today  on 
revenue  tomorrow. 

There  Is  an  Intimate  relation  between  linear  control  and  linear  programming. 
Any  problem  which  can  be  formulated  as  a  discrete-time  linear  control  problem 
with  a  finite  horizon  also  can  be  formulated  as  a  multi-stage  linear  programming 
problem  and  vice-versa.   Applications  Include  defense  [32],  production,     ' 
transportation  and  investment  [3A],  regional  and  national  economic  planning 

[35],  education  and  manpower  planning  [36],  and  water  resource  development  [37]. 

There  are,  however,  a   number  of  advantages  to  formulating  a  problem 
Involving  allocating  resources  over  time  as  a  problem  In  linear  control  rather 
than  as  a  problem  in  linear  programming.   The  formulation  of  dynamic  allocation 
problems  in  forms  of  state  and  control  variables  is  natural  since  the  variables 
have  immediate  physical  interpretation  in  terms  of  stocks  of  commodities  and 
levels  of  activities.   Moreover,  the  problem  need  only  be  formulated  for  the' 
representative  period  using  a  standard  format.  Alternatives  and  generalizations, 
even  to  include  non-linearities,  then  can  be  made  more  readily.   Also  the 
boundary  conditions  on  the  initial  and  terminal  stocks  of  commodities,  which 
are  Implicit  or  explicit  in  every  allocation  problem  are  more  apparent  in  the 
linear  control  formulation. 

Once  the  allocation  problem  is  formulated  As   a  linear  control  problem,  there 
are  a  number  of  ways  the  dual  maximum  principle  can  be  applied  to  the  solution 
of  the  problem.   The  linear  control  problem  can  be  reformulated  as  a  multi-stage 
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linear  programming  problem.   The  simplex  algorithm  and  its  variants  [38]  can, 
then,  be  considered  as  applications  of  the  dual  maximum  principle.   The  dis- 
advantage of  this  approach  is  that  the  resulting  linear  programming  problem 
may  become  so  large  that  it  is  intractable.   The  problem,  however,  does  have 
the  special  "staircase"  structure.  Illustrated  in  Figure  1,  to  which  we  can 
apply  the  decomposition  principle  [38].   The  maximum  principle  for  linear 
systems  may  even  be  considered  to  be  a  consequence  of  the  decomposition  prin- 
ciple of  linear  programming  [27].  The  usual  approach,  however,  is  to  use  the 
maximum  principle  to  express  the  optimal  control  as  a  function  of  the  state  and 
co-state  vectors.   The  disadvantage  of  this  approach  to  the  solution  of  linear 
control  problems  is  that  the  solution  to  the  Hamiltonlan  programming  problem 
may  not  be  unique. 

The  main  limitation  to  the  application  of  linear  control  theory  is  the 
assumption  of  convexity.   This  assumption  excludes  economies  of  mass  production 
and  discrete  changes  in  the  state  and  control  variables  [38].  There  are 
situations  in  which  the  absence  of  convexity  is  too  blatent  to  be  ignored. 
There  are  many' situations,  however,  in  which  the  assumption  of  convexity  is 
not  unreasonable. 
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CONCLUSIONS 


The  main  result  of  this  paper  is  a  dual  maximum  principle  for  discrete- 
time  linear  systems.  A  feasible  solution  to  the  primal  (dual)  linear  control 
problem  is  an  optimal  solution  If  and  only  If  there  exists  a  feasible  solution 
to  the  dual  (primal)  control  problem  with  the  primal  and  dual  Hamiltonians 
equal  and  satisfying  the  primal  and  dual  transversallty  conditions,  where  the 
dual  of  the  Hamlltonlan  programming  problem  associated  with  the  primal  linear 
control  problem  is  the  dual  of  the  Hamlltonlan  problem  associated  with  the 
dual  control  problem.  The  main  application  of  the  maximum  principle  in 
modern  control  theory  has  been  in  the  solution  of  optimal  control  problems. 
The  dual  maximum  principle  can  be  helpful,  however,  not  only  in  the  solution 
of  linear  control  problems,  but  also  in  their  formulation,  interpretation, 
and  analysis.  This  is  particularly  true  for  problems  Involving  allocation 
and  valuation.  ' 
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